Enable native Bazel builds and experimental CI cache dogfooding by Evangelink · Pull Request #12204 · microsoft/testfx

Amaury Levé (Evangelink) · 2026-11-04T13:06:04Z

Pull request

Summary

Add an opt-in native Bazel build for TestFx and a separate CI experiment to measure build-cache and test-result reuse over real changes. The existing MSBuild/Arcade pipeline remains the required validation and shipping authority; this does not replace it or claim complete shipping parity.

Compile the MSTest/MTP source graph directly with rules_dotnet, including modern framework variants, netstandard2.0/net462 compatibility, localization, analyzers/generators, and bounded Windows library/controller support. This is not an MSBuild wrapper; MSBuild is used separately for package consumers and the comparison baseline.
Produce experimental native framework/adapter/analyzer/SDK packages and run the existing unit suites, portable samples, browser bundle, and benchmark paths. Preserve public-only development signing, friend identities, reference aliases, and source-path mapping.
Add an independent Azure dogfood definition with cached, cold, forced-test, and matched MSBuild modes. It restores branch-scoped cache snapshots into fresh job directories, separates reused results from new executions, and refuses speedup claims when source, runtime, selection, outcomes, or timing evidence disagree.

Related issue: N/A; this is the user-requested staged Bazel experiment.

Design and decision record

Maintained research, alternatives, experiments, and limitations and commands/scope guide.

The default native toolchain remains SDK11, with the exact repository SDK12 available explicitly. The dogfood selects SDK12 and a net10.0 Release unit workload, preserving each suite's existing runtime layout. Cache transport initially uses Azure Cache@2; a remote action-cache service is deferred until transfer overhead justifies it. A parallel, non-required pipeline was chosen over a serial step in the existing build.

Important boundaries:

managed_all and the compatibility/Windows aggregates primarily materialize assembly outputs; they are not equivalent to a full localized shipping build. Dogfood builds the selected test targets and their runtime dependencies instead.
Public signing establishes development assembly identity, not official assembly/package signatures.
Test caching is per test binary, not per MSTest method. A versioned, opt-in identity policy normalizes only the four clock fields in one existing DynamicData display name; original test bodies, raw TRXs, and other data remain unchanged.
Native XAML/classic-UWP parity, the original net8 NativeAOT publisher path, complete acceptance/other-OS coverage, official signing, and release certification remain outside the delivered scope.

Demonstration

A text transcript is more useful than screenshots for this build/cache change. Observed locally during implementation:

Native SDK12 protocol run: 27 selected, 27 executed, 27 passed.
Fresh-output cache run:    27 selected, 0 newly executed, 27 reused.
                          453 build actions restored from the disk cache.
MSBuild protocol run:     The same 27 canonical identities and outcomes,
                          on the same runtime.

Cached TRXs are retained under reports/cached, never republished to Azure as new test executions. This demonstrates local reuse, not an Azure end-to-end speedup. Raw logs/binlogs/TRXs are retained locally and have not been uploaded as CI artifacts.

Validation and regression proof

Final helper source is committed in 8b4fd8b581d34b66c78f477f90b6817e9a58f7d8. The following standalone commands passed immediately before committing:

Exact command	Result
`.\eng\bazel\test-dogfood-results.ps1`	Exit 0; 69/69 standalone parser checks passed.
`.\eng\bazel\test-dogfood-runner.ps1`	Exit 0; 4/4 launcher/lifetime checks passed.
`.\eng\bazel\test-dogfood-comparison.ps1`	Exit 0; 5/5 comparison guards passed.
`git diff --cached --check`	Exit 0. Both new Azure YAML files also parsed successfully; this is not Azure service-side compilation.

An additional eight retained-report checks passed, giving 77 parser checks when those fixtures are included. The finalized parser/test file hashes were read back before commit.

The composed dogfood runs below were performed on implementation working states based on 0dcfbd1c2d9724e4891a042c357b3554d39cee30, not as a complete final-head rerun. Their records retain exact patch hashes. The two full runs share patch hash 902AE5FC99A2AABCBB55FEA18622B9FED44806BE28291DE2AF9AA47DF6341202.

For the commands below, the recorded path values were:

$e = Join-Path $PWD 'artifacts\bazel-dogfood-validation\20261104-104557-177b20c8'
$c = 'Q:\tfbc\dogfood-20261104-104557-177b20c8'

Exact effective command	Result
`.\eng\bazel\dogfood.ps1 -Mode Cached -RunRoot (Join-Path $e 'native-hit') -CacheRoot $c -Targets '//:protocol_contract_tests'`	Exit 0; 27 cached passes, 453 disk-cache build hits, fresh output root.
`.\eng\bazel\dogfood.ps1 -Mode MSBuild -RunRoot (Join-Path $e 'msbuild-one3') -CacheRoot $c -Targets '//:protocol_contract_tests'`	Exit 0; 27 executed passes, matching native identities/outcomes/runtime.
`.\eng\bazel\dogfood.ps1 -Mode Cached -RunRoot (Join-Path $e 'native-full2') -CacheRoot $c`	Exit 1, preserving child exit 3; all 13 suites selected 12,505 cases: 12,487 passed, 15 skipped, 3 failed.
`.\eng\bazel\dogfood.ps1 -Mode MSBuild -RunRoot (Join-Path $e 'msbuild-full') -CacheRoot $c`	Exit 1; 12,505 cases: 12,479 passed, 15 skipped, 11 failed. Nine failures exposed a missing isolated legacy-task fixture; the same two Win32 failures remained.
`.\eng\bazel\dogfood.ps1 -Mode MSBuild -RunRoot (Join-Path $e 'msbuild-legacy') -CacheRoot $c -Targets '//:msbuild_tests'`	Exit 0; after staging the existing netstandard2.0 sidecar contract, all 388 cases passed. No original tests were changed.

The native failures were two previously observed Win32-1346 impersonation failures and FailedLaunchDisposesProcessHandles observing 13 handles against an upper bound of 12. The handle case passed in the MSBuild run; no assertion, skip, or retry was changed to hide it. All 13 selections reconcile after the narrow clock-display projection.

Earlier producer/package validation at c910ab89ce7dc95c3ab2666ed62c4172a9950183 is recorded with exact commands and consumer provenance in the decision record: SDK/meta 40 steps passed, MSTest/runtime/satellite sensitivity 31 steps passed, analyzer package 15 cases passed, plus real Windows activation, browser exits 0/9, and portable samples. These are historical local results, not final-head CI certification.

Still unverified: registration/execution of the new Azure pipeline, real cross-agent Cache@2 restore/upload timings, a complete post-correction four-mode comparison, and the planned source/test/resource/configuration invalidation experiment. No percentage CI saving is claimed. The required pipelines remain unchanged and authoritative.

Test-value evidence

Requirement	Evidence
Missing test evidence cannot become green	`BepRejectsMissingExpectedTarget`: with the original 67-case suite unchanged, removing only the missing-target guard produced 66 passes and the intended failure (exit 1); restoring it produced 67 passes (exit 0).
Clock normalization is narrow and opt-in	`TrxTimingPolicyDefaultsToExact`, `TrxTimingPolicyPreservesDurations`, and `NativeTrxTimingPolicyMatchesPairedSelections`: the finalized 77-case parser suite passed; disabling only normalization in an isolated copy produced 70 passes and seven intended assertion failures; restoring it returned 77 passes. The real paired 3,380-case selections changed exactly one key each.
Cache/control accounting stays honest	`MatchedControlsMeasureNetJobSavingsAndKeepReuseSeparate`, `EqualCountsDoNotHideDifferentTestIdentities`, `ForcedExecutionControlRejectsCachedTests`, and `IncompleteTimelineRemainsUnknownRatherThanZero` passed in the five-case comparison suite.
Child invocation preserves the intended command and failure	`ScriptPreservesColonBearingBazelOptions`, `WindowsPowerShellPreservesNamedBuildParameters`, `ProcessPreservesFailureExitAndBothLogs`, and `ProcessStreamsLogsBeforeExitAndJoinsTheChild` passed.

The normalization counterfactual patch SHA-256 is 3fd457014af48ab69e8f196a1a87b3e9f69e7f4e463eb2a2355c59e2f6c603c2. Raw reports and original unit sources were preserved. Broader cache-invalidation sensitivity remains unproven, as called out above.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Adopt the immutable upstream SDK-discovery rules, build the portable .NET 8-11 matrix, add existing test suites, and validate native MTP NuGets and self-contained runtime layouts without changing the MSBuild shipping pipeline. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Use preview.bazel identities compatible with the pinned CodeCoverage minimum while remaining prerelease, and allow per-run evidence roots and build concurrency for coordinated worktrees. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Preserve global and named-only reference semantics across all compiler passes, reject invalid alias declarations, and record the resolved producer SDK in validation evidence. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 6d95203b168af0022da48ee8f5e71532c372de13)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 6f69c9d135c542f6c1f3bacf4e003da66506e9f3)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit a29a5ffc0b1005848e71a645abcdeddb818113ff)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit b156a7381a6b627d504e2d2b06e4002c6d3d97e4)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit f8d9b6a7cd524634b57363aba3c244724695edfa)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 46e50f3636cc8a7fcd3c5918b2c204a1282243ab)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 31cb48aaa42618429ba8f6e1217d392bffd3b51b)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 7751a9a6bf21d972f583f84a353219a2116360fb)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 9e1604807cacd4f704b4e416137a8ad106513634)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 6ade9ed036b19acf948f498ebe9d886203715a9c)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 09318a2a1b35b74e2704e96b6843ae59d3fb768f)

Query the declared consumer toolchain explicitly, verify its unique SDK archive and executable provenance, and avoid requiring consumer SDK files in the producer action forest. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 6d851a91cf44d479ebf549996b337c26849f4578)

Consolidate analyzer metadata with the verified netstandard compatibility attribute while retaining all 15 analyzer package and compiler/legacy-host conformance checks. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Declare explicit portable framework and adapter payloads plus their real telemetry and MSBuild support closure. Keep existing MTP package targets unchanged. Native execution awaits the coordinated analyzer, netstandard task, and preview.bazel prerequisites; the decision record preserves those gates. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 983ec71b2be96d9bd5863101ae50a263d6188972)

Remove exact inherited MSBuild environment entries and restore their original presence and value. Record the imported netstandard prerequisite and keep full package execution gated on remaining handoffs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 88326266d553c321b107631c8c6b0ac7d789696c)

Compare the four source-declared display names and discovery total, rather than expecting fully qualified names that the native reporter does not print. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 8fa24f9e363915f7224b40b7829bf7c73bc5a067)

Mirror the MSBuild task project's RID-specific runtime payload without packing reference-only host assemblies. Clear inherited NuGet fallback folders and use the explicit consumer restore configuration. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 450b4f78980349a63a6dc01fbf7063c77633298a)

Declare the already-resolved platforms 1.0.0 module directly so the shipping Windows dependency condition is visible to root package rules. Record the separate compatibility source-selector blocker and inspected native framework package. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 7b1f9159c6e79e9cf5c875c77cb35f52a3559a8a)

Generate literal dependency ranges in the native pack request because the upstream versionFile contract accepts only individual versions. All eight MSTest closure archives now build with the corrected preview.bazel identities and preserved bounds; composed consumer checks remain the next gate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit ba725abc5cb28aaa36ec789ab50891f5e65b0b6e)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit c80107ee1484473b87b449bc082daed08ec7a9d6)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Read the failed test's ErrorInfo message rather than XmlDocument.InnerText, which is null in PowerShell. The guard is checked against the actual intentional-failure report and rejects an unrelated marker. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 9e6075e6dc2fd3faa1675ca89f9f8298d8be8578)

Use MSBuild's percent escaping for embedded quotation marks. The actual native task now executes the spaced-path passing and intentionally failing consumer with parent exits 0 and 1. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 42dac12d6c5d7c79d1d6b1aed2fd20bc0683396a)

Keep the read-only original adapter out of the mutation output slot. The complete isolated-feed validation passed all 31 expected-exit steps across actual .NET 8/9/10/11 runtimes, native MSBuild task execution, and the same-test satellite-removal red/green comparison. Preserve unsigned evidence while the coordinator restores shared public-signing compatibility. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 9648087e900453bc84a5c004c2f5d2e0f1be8d32)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 870a6d86b6952e6ed2eb84411c34f92c3c7ee8ad)

Keep raw logs intact, normalize only the assertion view, and force colored negative probes so hosted console behavior is covered locally. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions

🧪 Test quality review — PR #12204

Compared merge base 92e83edafb54811009aae479caf29a763f19a99a with head 03e3d3503573ab52dd7b97e2a6f6f3b9d929d21b. Evidence applies to this head, not every intermediate commit.

Test-value evidence

All evidence is static or unverified. This workflow runs no tests. The author's PR-body claims (for example the 67→66 and 77→70 counterfactual runs) concern the dogfood PowerShell parser suites, which are not in this PR's changed test files. I have no accessible paired execution record for them.

Bazel build tools (resx/xlf/metadata/package version): BuildToolsTests asserts exact generated output (BOM, raw-string source, resource values, exit codes and error text) for valid and rejected input, and that no output file is written on rejection. Prediction: newline/BOM regressions or a dropped validation would likely fail. Static prediction only.
SDK package generation: SdkPackageTests covers template/version preservation and rejection of contract drift. Static prediction only.
Reference aliases: AliasTests covers alias identity, friend access and public key token. Static prediction only.
EqtTraceLoggerContinuationTests: the isolated child host is now launched from Environment.ProcessPath or the test assembly instead of .dotnet/dotnet test. This is a behavior-preserving harness change. Both revisions would be expected to pass, but this is unverified here.
~100 Bazel/Starlark/PowerShell/CI/docs files: no tests were found for the .bzl rules or .ps1 scripts in this diff. These contracts are not assessed and remain unverified.

🧵 Parallel safety

STATUS: PARTIAL. I did not launch the specialist subtask. A direct check shows the new tests use per-test temp directories. EqtTraceLoggerContinuationTests launches a child host with a unique results directory. I performed no assembly-level parallelization analysis, and one config candidate was not analyzed. No findings come from the direct check, and this does not mean parallel safety is clean.

Per-test assessment

Pseudo-mutation evidence is not provided. Grading was a static review of the changed tests. Rows are grouped by test, with data-driven cases combined.

<table>
<thead>
<tr><th>Result</th><th>Quality</th><th>Test</th><th>Notes</th><th>How to improve</th></tr>
</thead>
<tbody>
<tr>
<td>Pass</td>
<td>A (90–100)</td>
<td>new <code>BuildToolsTests.<br>Resx_<br>PreservesTextAndEmitsBom</code></td>
<td>Exact output, BOM and values are asserted for LF and CRLF.</td>
<td>None</td>
</tr>
<tr>
<td>Pass</td>
<td>B (80–89)</td>
<td>new <code>BuildToolsTests.<br>Xlf_, Resx_, Metadata_*</code></td>
<td>Exit codes, error text and absent outputs are asserted.</td>
<td>None</td>
</tr>
<tr>
<td>Pass</td>
<td>B (80–89)</td>
<td>new <code>SdkPackageTests.<br>*</code></td>
<td>Template, dependency and drift-rejection contracts are asserted.</td>
<td>None</td>
</tr>
<tr>
<td>Pass</td>
<td>B (80–89)</td>
<td>new <code>AliasTests.<br>*</code></td>
<td>Alias identity, friend access and public key token are asserted.</td>
<td>None</td>
</tr>
<tr>
<td>Pass</td>
<td>B (80–89)</td>
<td>mod <code>EqtTraceLoggerContinuationTests.<br>TraceBridgePersistsEverySeverityInAnIsolatedHost</code></td>
<td>The child must exit 0 with an isolated results directory.</td>
<td>None</td>
</tr>
</tbody>
</table>

<sub>This advisory review is heuristic and non-blocking. No inline findings were published.

Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~34 AIC · ⌖ ~1 AIC · ⊞ 8K

Copilot

🔵 Needs a closer look

The workflow trigger omits Directory.Packages.props, allowing central Roslyn version changes to bypass dependency-alignment validation and merge with stale native Paket pins.

2 open issues

_{Review effort: Balanced}

Run the Bazel drift guard for package-only edits and preserve relative result directories across parent/child working directories. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions

🧪 Test quality review — PR #12204

Compared merge base 92e83edafb54811009aae479caf29a763f19a99a with head 3504d985dd3483c2bd25c4ddc440bb71f23ccc8c. Evidence applies to this head only.

Test-value evidence

Static, bounded pass only. No tests, builds, or mutations were run, and no paired execution records were available. Everything below is unverified / static prediction, not executed red-green proof.

Bazel build/packaging infrastructure (about 100 new eng/bazel/**, .bzl, .ps1, docs, and workflow files): not assessed individually because of review budget. New unit tests cover only the C# build tools (Bazel.BuildTools.UnitTests, Bazel.SdkPackages.UnitTests) and the reference aliases (Bazel.ReferenceAliases.UnitTests). Starlark, PowerShell, and pipeline changes are unassessed.
EqtTraceLoggerContinuationTests: the child test host is now launched directly from Environment.ProcessPath (or the assembly on .NET Framework) instead of dotnet test --project. Static prediction: a broken child launch or a non-zero exit would still fail child.ExitCode.Should().Be(0, ...). The exact-revision execution is unverified.
Behavior-preserving or non-executable changes: not applicable.

🧵 Parallel safety

PARALLEL-SAFETY RESULT
STATUS: PARTIAL
NOTES: The parallel-safety specialist was not run because of review budget. The changed configuration candidate (BuildToolsTests.cs) and the new test assemblies were not assessed. This is not a clean result.
END PARALLEL-SAFETY RESULT

Per-test assessment

Per-test grading was not performed (review budget), so no grades are claimed. Changed test methods in scope, all Uncertain with no quality assigned:

<table>
<thead>
<tr><th>Result</th><th>Quality</th><th>Test</th><th>Notes</th><th>How to improve</th></tr>
</thead>
<tbody>
<tr><td>Uncertain</td><td>—</td><td>new <code>BuildToolsTests.<br>*</code> (6 methods)</td><td>Not graded; the tests exercise build tools.</td><td>Run the grading skill with the production tool sources.</td></tr>
<tr><td>Uncertain</td><td>—</td><td>new <code>SdkPackageTests.<br>*</code> (3 methods)</td><td>Not graded.</td><td>Run the grading skill with the production tool sources.</td></tr>
<tr><td>Uncertain</td><td>—</td><td>new <code>AliasTests.<br>*</code> (5 methods)</td><td>Not graded.</td><td>Run the grading skill with the alias fixtures.</td></tr>
<tr><td>Uncertain</td><td>—</td><td>mod <code>EqtTraceLoggerContinuationTests.<br>TraceBridgePersistsEverySeverityInAnIsolatedHost</code></td><td>Child launch now uses the host process directly.</td><td>Confirm the child run executes on both TFMs.</td></tr>
</tbody>
</table>

No inline findings were published; no verified actionable defect was identified.

_{This advisory review is heuristic and non-blocking. Re-run with /review-tests, or only the parallel-safety specialist with /parallel-audit.}

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~28 AIC · ⌖ ~1 AIC · ⊞ 8K

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

🔵 Needs a closer look

Human assessment is needed to determine whether Cold-only cache uploads skew the whole-job timing comparison and overstate reported savings.

No open issues 🎉

2 resolved since last review

_{Review effort: Balanced}

Link the stopwatch abstractions and preserve upstream child-fixture dispatch and lifetime in the native test entry point. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions

🧪 Test quality review — PR #12204

Compared merge base 6876cf113dce6e1db55092779e73840697a0af73 with head 3431ca74f3196a39675874032557c08761429aa8. Evidence applies to this head, not every intermediate commit.

Test-value evidence

This is a bounded static review. Nothing was built or executed, and no paired red/green execution records were available. Every result below is a static prediction or unverified.

Bazel build tooling tests (BuildToolsTests, SdkPackageTests, AliasTests): these are new tests for new tooling. They cover RESX/XLF handling, version templating, package-version selection, SDK package generation, drift rejection, and reference-alias identity. The names and visible assertions match the claimed contracts. Execution evidence is unverified, and the tests are not graded in detail.
EqtTraceLoggerContinuationTests.TraceBridgePersistsEverySeverityInAnIsolatedHost (modified): the test now starts the child with the current host (Environment.ProcessPath or the assembly). Before, it ran dotnet test from .dotnet/. The decisive assertion is still child.ExitCode.Should().Be(0, ...), plus the existing persisted-severity checks in the child. The refactor is expected to pass before and after, but this is unverified.
Non-test changes (.bzl, .ps1, MODULE.bazel, docs, pipelines, patches): not applicable or not assessed. This is a build-infrastructure change and I did not assess it in this run.

🧵 Parallel safety

PARALLEL-SAFETY RESULT: STATUS: PARTIAL. The specialist analysis was not completed in this run because of the size of the PR (115 files). This is not a clean result. Please run /parallel-audit.

Per-test assessment

Detailed per-test grading was not completed, because this PR adds about 600 lines of new tests plus a modified process-spawning test. The scorecard below therefore reports Uncertain for every group. It reflects missing evidence, not a defect. No inline findings were published. Existing review threads could not be checked for deduplication, so inline publication was withheld.

<table>
<thead>
<tr><th>Result</th><th>Quality</th><th>Test</th><th>Notes</th><th>How to improve</th></tr>
</thead>
<tbody>
<tr>
<td>Uncertain</td>
<td>—</td>
<td>new <code>BuildToolsTests.</code>, <code>SdkPackageTests.</code>, <code>AliasTests.*</code></td>
<td>Per-method grading was not completed in this bounded run.</td>
<td>Re-run <code>/review-tests</code> on a narrower scope for grades.</td>
</tr>
<tr>
<td>Uncertain</td>
<td>—</td>
<td>mod <code>EqtTraceLoggerContinuationTests.<br>TraceBridgePersistsEverySeverityInAnIsolatedHost</code></td>
<td>Child host selection was refactored; the exit-code assertion is kept.</td>
<td>Provide before/after execution results for this test.</td>
</tr>
</tbody>
</table>

<sub>This advisory review is heuristic and non-blocking.

Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~22 AIC · ⌖ ~1 AIC · ⊞ 8K

Copilot

🔵 Needs a closer look

The final revision lacks a complete Azure four-mode run validating the native toolchain, cross-agent cache reuse, and input invalidation, leaving key execution paths unverified.

No open issues 🎉

_{Review effort: Balanced}

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions

🧪 Test quality review — PR #12204

Compared merge base 6876cf113dce6e1db55092779e73840697a0af73 with head d45cf09562da4281be48831472cd5227117bef76. Evidence applies to this head, not every intermediate commit.

Test-value evidence

Static, partial assessment only. No tests were executed, and the PR's executed-evidence claims were not independently verified at this head.

Bazel build/test infrastructure (115 files, ~21k lines): this is non-shipping, opt-in tooling. Only the new Bazel.BuildTools.UnitTests suite was examined in detail: the resx and xlf conversion cases and the invalid-catalog DataRow cases (culture, stale, duplicate, inline). They assert exit code, exact resource values, and that no output file is written. Static prediction: a regression in those behaviors would likely be caught. Unverified.
EqtTraceLoggerContinuationTests: the child-host launch changed from dotnet test to a direct host invocation. The child.ExitCode.Should().Be(0, ...) assertion is retained. Static prediction: the isolation contract is still checked. Unverified.
Not assessed: most .bzl/PowerShell/Azure pipeline files and the Bazel.SdkPackages and Bazel.ReferenceAliases tests (review budget). The PR body's red/green claims for the dogfood parser suites are author-reported and are not accepted here as paired execution proof.

🧵 Parallel safety

STATUS: PARTIAL. The parallel-safety specialist was not run in this pass, so there are no findings and no clean verdict. BuildToolsTests.cs was flagged as a parallelization candidate. It uses per-test Write/PathFor helpers with a TestContext constructor, which suggests per-test paths, but the helper bodies were not verified.

Per-test assessment

Pseudo-mutation evidence below consists of static predictions, not executed mutation results.

<table>
<thead>
<tr><th>Result</th><th>Quality</th><th>Test</th><th>Notes</th><th>How to improve</th></tr>
</thead>
<tbody>
<tr>
<td>Pass</td>
<td>A (90–100)</td>
<td>new <code>BuildToolsTests.<br>Resx_<br>PreservesTextAndEmitsBom</code></td>
<td>Exact values, BOM and generated source are asserted for LF and CRLF.</td>
<td>None</td>
</tr>
<tr>
<td>Pass</td>
<td>A (90–100)</td>
<td>new <code>BuildToolsTests.<br>Xlf_<br>AppliesTargetsAndPreservesMissingTargetFallback</code></td>
<td>Each translation state is asserted against its expected value.</td>
<td>None</td>
</tr>
<tr>
<td>Pass</td>
<td>A (90–100)</td>
<td>new <code>BuildToolsTests.<br>Xlf_<br>RejectsInvalidCatalog</code></td>
<td>Per-scenario error text, nonzero exit and no output file are asserted.</td>
<td>None</td>
</tr>
<tr>
<td>Pass</td>
<td>A (90–100)</td>
<td>new <code>BuildToolsTests.<br>Resx_<br>RejectsNonStringDataBeforeWritingOutputs</code></td>
<td>Error text and absence of both outputs are asserted.</td>
<td>None</td>
</tr>
<tr>
<td>Uncertain</td>
<td>—</td>
<td>new remaining tests in <code>BuildToolsTests</code>, <code>SdkPackageTests</code>, <code>AliasTests</code></td>
<td>Not graded in this pass because of review budget.</td>
<td>Re-run <code>/review-tests</code> for per-test grades.</td>
</tr>
<tr>
<td>Uncertain</td>
<td>—</td>
<td>mod <code>EqtTraceLoggerContinuationTests.<br>TraceBridgePersistsEverySeverityInAnIsolatedHost</code></td>
<td>The child-host command changed; production and child behavior were not traced.</td>
<td>Confirm the child run on net462 and on net10.0 under the dotnet host.</td>
</tr>
</tbody>
</table>

No inline comments were queued. Existing-thread and prior-review lookups were not completed, so duplicate suppression is unconfirmed.

<sub>This advisory review is heuristic and non-blocking.

Re-run the combined review with /review-tests, or only the parallel-safety specialist with /parallel-audit.</sub>

🤖 Automated content by GitHub Copilot. Generated by the Test Quality Review on PR (on open / sync) workflow. · copilot · auto · ~31 AIC · ⌖ ~1 AIC · ⊞ 8K

Copilot

🔵 Needs a closer look

Human assessment is needed because Cold-only action-cache uploads inflate its full job duration and may overstate the apparent savings from cached modes.

No open issues 🎉

_{Review effort: Balanced}

Amaury Levé (Evangelink) · 2026-11-05T16:16:19Z

/azp run microsoft.testfx

azure-pipelines · 2026-11-05T16:16:40Z

Azure Pipelines: Successfully started running 1 pipeline(s).

github-actions · 2026-11-05T18:06:17Z

Final test-triage resolution — Azure Pipelines build 1666178

Result: inconclusive (no actionable test defect found; no issue warranted).

The build (microsoft.testfx, 20261105.29) completed as succeeded. The timeline has no failed legs, crashes, hangs, or diagnostics. The only warning is low free memory on the "Test (unit tests only)" step.
The collected reports contain 101,579 passed, 835 skipped and 6 "failed" records. All 6 come from Windows_App_Model_Diagnostics_Attempt1 fixture/probe outputs: intentional probes under extension-probes, NativeSidecar, and PackagedWinUI/Retries. Their messages are deliberate, e.g. "This probe deliberately fails the first retry attempt", "expected host failure", and "The first AppContainer host fails so Retry must activate a second host". They come from retry/native-sidecar scenario tests, not product regressions.
No slowness regressions were found (0 of 30 slow candidates met the p95 and 60s floor criteria). No flakiness was proven (no CTRF fail-then-pass records).
Caveat: evidence collection was incomplete (evidenceIncomplete=true, 1 artifact download failure), so this is not a claim that all legs are clean. Re-run or inspect the failed artifact download if a missing leg is a concern.

Confidence: high that the 6 failed records are not real failures; moderate overall because of the missing artifact.

🤖 Automated content by GitHub Copilot. Generated by the Pipeline Test Triage workflow. · copilot · auto · ~27 AIC · ⌖ ~1 AIC · ⊞ 10K

Amaury Levé (Evangelink) and others added 30 commits October 29, 2026 14:47

Enable staged native Bazel builds and validation

1ec3737

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Align Bazel test tuple names with repository conventions

22a3f9d

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Package MSTest analyzers natively with Bazel

7bc60a3

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 6d95203b168af0022da48ee8f5e71532c372de13)

Validate analyzer package preview identities and isolated consumers

bb3d555

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 6f69c9d135c542f6c1f3bacf4e003da66506e9f3)

Add native netstandard platform and MSBuild targets

67046b4

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit a29a5ffc0b1005848e71a645abcdeddb818113ff)

Emit exact framework metadata for native compatibility assemblies

05c7c92

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit b156a7381a6b627d504e2d2b06e4002c6d3d97e4)

Record analyzer package counterfactual evidence

112c626

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit f8d9b6a7cd524634b57363aba3c244724695edfa)

Preserve no-op removals in native JSON source selection

325eca5

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 46e50f3636cc8a7fcd3c5918b2c204a1282243ab)

Add native MSTest SDK and meta-package generation

1a4d289

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 31cb48aaa42618429ba8f6e1217d392bffd3b51b)

Align native SDK package prerelease versions

256c9f2

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 7751a9a6bf21d972f583f84a353219a2116360fb)

Isolate inherited MSBuild executable selection in SDK consumers

0a1443c

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 9e1604807cacd4f704b4e416137a8ad106513634)

Record native SDK package and sensitivity evidence

9036c23

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 6ade9ed036b19acf948f498ebe9d886203715a9c)

Expand SDK consumer contract and producer provenance checks

11ae3b9

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 09318a2a1b35b74e2704e96b6843ae59d3fb768f)

Verify native SDK and meta-package consumer boundaries

960366e

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit c80107ee1484473b87b449bc082daed08ec7a9d6)

Preserve native public signing and friend assembly identities

aff2cf0

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Add native netstandard and net462 compatibility variants

3a3d939

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> (cherry picked from commit 870a6d86b6952e6ed2eb84411c34f92c3c7ee8ad)

microsoft-github-policy-service Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 5, 2026

github-actions Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 5, 2026

Handle ANSI-styled output in native Bazel validation

03e3d35

Keep raw logs intact, normalize only the assertion view, and force colored negative probes so hosted console behavior is covered locally. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI balanced review requested due to automatic review settings November 5, 2026 10:34

github-actions Bot added state/needs-review Awaiting review from the team. and removed state/needs-review Awaiting review from the team. labels Nov 5, 2026

Copilot started reviewing on behalf of Amaury Levé (Evangelink) November 5, 2026 10:34

View session

github-actions Bot reviewed Nov 5, 2026

View reviewed changes

microsoft-github-policy-service Bot removed the state/needs-review Awaiting review from the team. label Nov 5, 2026

Copilot AI reviewed Nov 5, 2026

View reviewed changes

Cover central package changes and normalize trace child result paths

3504d98

Run the Bazel drift guard for package-only edits and preserve relative result directories across parent/child working directories. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions Bot reviewed Nov 5, 2026

View reviewed changes

Merge current main for native server-client validation

dbcd61c

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-code-quality Bot found potential problems Nov 5, 2026

View reviewed changes

Comment thread test/UnitTests/MSTestAdapter.UnitTests/EqtTraceLoggerContinuationTests.cs Resolved

Copilot AI reviewed Nov 5, 2026

View reviewed changes

Align Bazel server-client sources and fixtures with main

3431ca7

Link the stopwatch abstractions and preserve upstream child-fixture dispatch and lifetime in the native test entry point. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions Bot reviewed Nov 5, 2026

View reviewed changes

Copilot AI reviewed Nov 5, 2026

View reviewed changes

Amaury Levé (Evangelink) mentioned this pull request Nov 5, 2026

Fix table rendering in test-quality reviews #12222

Merged

This comment has been minimized.

Sign in to view

Fix heading spacing in Bazel validation notes

d45cf09

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-actions Bot reviewed Nov 5, 2026

View reviewed changes

Copilot AI reviewed Nov 5, 2026

View reviewed changes

YuliiaKovalova approved these changes Nov 6, 2026

View reviewed changes

Amaury Levé (Evangelink) mentioned this pull request Nov 6, 2026

Reuse Bazel build caches in GitHub Actions #12253

Merged

Navigation Menu

Repository navigation

Enable native Bazel builds and experimental CI cache dogfooding - #12204

Enable native Bazel builds and experimental CI cache dogfooding#12204
Amaury Levé (Evangelink) merged 62 commits into
mainfrom
dev/amauryleve/bazel-build-migration

Amaury Levé (Evangelink) commented Nov 4, 2026

Uh oh!

github-actions Bot left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

This comment has been minimized.

github-actions Bot left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Amaury Levé (Evangelink) commented Nov 5, 2026

Uh oh!

azure-pipelines Bot commented Nov 5, 2026

Uh oh!

github-actions Bot commented Nov 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Amaury Levé (Evangelink) commented Nov 4, 2026

Pull request

Summary

Design and decision record

Demonstration

Validation and regression proof

Test-value evidence

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🧪 Test quality review — PR #12204

Test-value evidence

🧵 Parallel safety

Per-test assessment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🧪 Test quality review — PR #12204

Test-value evidence

🧵 Parallel safety

Per-test assessment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

🧪 Test quality review — PR #12204

Test-value evidence

🧵 Parallel safety

Per-test assessment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

This comment has been minimized.

github-actions Bot left a comment

Choose a reason for hiding this comment

🧪 Test quality review — PR #12204

Test-value evidence

🧵 Parallel safety

Per-test assessment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Amaury Levé (Evangelink) commented Nov 5, 2026

Uh oh!

azure-pipelines Bot commented Nov 5, 2026

Uh oh!

github-actions Bot commented Nov 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants